Decorating surfaces with textures

ABSTRACT

Surfaces can be decorated with texture tiling and/or texture painting using one or more sample textures, such as BTFs. In a described implementation of texture tiling, a patch-based BTF synthesis algorithm is utilized. In an example embodiment, a mesh of a target surface is re-sampled to produce a dense mesh such that there is a one-to-one correspondence between vertices of the dense mesh and pixels of an image. Patch matching is then employed with a working image that corresponds to a working patch. In an example embodiment, the patch matching is effectuated using translations and rotations of the sample texture. In a described implementation of texture painting, irregular feature(s) of a sample texture are synthesized onto a target surface. In an example embodiment, user-specified constraints as to a desired foreground feature of a sample texture and an intended foreground region of a targeted surface are factored into a graphcut operation.

TECHNICAL FIELD

This disclosure relates in general to decorating surfaces with texturessuch as bidirectional texture functions (BTFs) and in particular, by wayof example but not limitation, to (i) tiling regular BTF samples onto atarget surface and/or (ii) painting irregular BTF samples onto a e.g.tiled surface.

BACKGROUND

Texture mapping was introduced about three decades ago as a way to addsurface detail without adding geometry. Texture-mapped polygons havesince become the basic primitives of the standard graphics pipeline.Unfortunately, texture-mapped surfaces have a distinctive look that setsthem apart from reality. For example, they cannot accurately respond tochanges in illumination and viewpoint.

Real-world surfaces are usually not smooth; instead, they are coveredwith textures that arise from both spatially-variant reflectance andfine-scale geometry details that are often termed “mesostructures”. Realsurfaces also exhibit imperfections (e.g., dirt, cracks, and scratches)that typically result from relatively complicated physical processes.Even three decades later, capturing these surface characteristics is achallenging goal for computer graphics.

Accordingly, there is a need for schemes, mechanisms, techniques, etc.that can efficiently and/or conveniently enable computer graphics tobetter capture spatially-variant reflectance and/or fine-scale geometricmesostructures.

SUMMARY

Surfaces can be decorated with one or more sample textures, such asbidirectional texture functions (BTFs). The decorating can involvetexture tiling and/or texture painting. In a described implementation oftexture tiling, a patch-based BTF synthesis algorithm is utilized. In anexample embodiment, a mesh of a target surface is re-sampled to producea dense mesh such that there is a one-to-one correspondence betweenvertices of the dense mesh and pixels of an image. A working patch isfound. Patch matching is then employed with a working image thatcorresponds to the working patch. In an example embodiment, the patchmatching is effectuated using both translations and rotations of thesample texture. Patch fitting is then effectuated with the correspondingworking patch. In a described implementation of texture painting,irregular feature(s) of a sample texture are synthesized onto a targetsurface. In an example embodiment, user-specified constraints as to thedesired foreground feature of a sample texture and the intendedforeground region of a targeted surface are factored into a graphcutoperation. For instance, defined constraint energy may be minimized aspart of the graphcut operation.

Other method, system, approach, apparatus, device, media, procedure,API, arrangement, etc. implementations are described herein.

BRIEF DESCRIPTION OF THE DRAWINGS

The same numbers are used throughout the drawings to reference likeand/or corresponding aspects, features, and components.

FIG. 1 is a block diagram illustration of an example surface decoratorthat is capable of surface tiling and/or surface painting.

FIG. 2 illustrates example blocks relating to surface tiling with aregular texture sample and surface painting with an irregular texturesample.

FIG. 3 is a flow diagram that illustrates an example of a method fortiling a surface with a regular texture sample.

FIG. 4 is a flow diagram that illustrates an example of a method forpainting a surface with an irregular texture sample.

FIG. 5A illustrates an example of a surface that has been tiled with arelatively realistic texture.

FIG. 5B illustrates an example of the tiled surface of FIG. 5A afterbeing augmented with an irregular feature.

FIGS. 6A-6F illustrate an example surface mesh and sample BTF in thecontext of an example of a straightforward extension of graphcuts topattern mapping as compared to an example of results of a GIM-basedsampling approach.

FIGS. 7A-7D illustrate an example construction of a dense mesh from aninput mesh for subsequent BTF synthesis.

FIGS. 8A-8D illustrate an example BTF synthesis using a work patch and awork image.

FIG. 9 illustrates an example SVD factorization of a BTF sample.

FIGS. 10A-10D illustrate the construction or painting of a constrainedquilt with an irregular feature for a foreground texture.

FIG. 11 illustrates a graphical formulation of a seam finding problemfor graphcut texture painting.

FIG. 12 illustrates an input texture sample and resulting decoratedsurfaces under different example constraint function m₀ scenarios.

FIG. 13 illustrates an example of a computing (or general device)operating environment that is capable of (wholly or partially)implementing at least one aspect of decorating surfaces with textures asdescribed herein.

DETAILED DESCRIPTION Introduction

Schemes, mechanisms, techniques, etc. for decorating arbitrary surfaceswith bidirectional texture functions (BTFs) are described herein. In adescribed implementation, BTF-decorated surfaces are generated in twosteps. First, a BTF is automatically tiled or synthesized over a targetsurface from a given BTF sample. Second, the user is enabled tointeractively paint BTF patches onto the tiled surface, such that thepainted patches seamlessly integrate with the background patterns.

In a described implementation, a patch-based texture synthesis approachknown as quilting is employed. More specifically, a graphcut algorithmfor BTF tiling or synthesis on surfaces is employed. The describedgraphcut algorithm works well for a wide variety of BTF samples,including at least some of those that present problems for existingalgorithms. In another described implementation, a graphcut texturepainting algorithm for creating new surface irregularities such asimperfections (e.g., dirt, cracks, scratches, etc.) from existingirregularities found in input BTF samples is employed.

Using the algorithms described herein, surfaces can be decorated withreal-world textures that have spatially-variant reflectance, fine-scalegeometry details, and/or surfaces imperfections. For example, BTF tilingenables a BTF to be synthesized onto a target surface of a geometricmodel, and BTF painting allows imperfections of real materials to becaptured and then painted onto geometric models, including those modelsthat have already been tiled.

In the following section, decorating surfaces with texture samples isdescribed qualitatively with reference to FIGS. 1-4. In the sectionthereafter, decorating surfaces with texture samples is describedquantitatively with particular reference to FIGS. 5A-12.

Qualitative Description of Decorating Surfaces with Texture Samples

FIG. 1 is a block diagram illustration of an example surface decorator106 that is capable of surface tiling 106(T) and/or surface painting106(P). As illustrated, surface decorator 106 includes a surface tiler106(T) and a surface painter 106(P). Surface decorator 106, surfacetiler 106(T), and surface painter 106(P) may be realized individually orjointly using hardware, software, firmware, some combination thereof,and so forth. For example, they may be realized as modules of a softwareprogram. Alternatively, all or part of their functionality may berealized as a hard-coded semiconductor or chipset.

In a described implementation, surface tiler 106(T) is capable ofapplying a surface texture to all or a part of a target surface 102.Surface tiler 106(T) synthesizes a relatively homogenous texture thatcan still maintain spatially-variant reflectance, mesostructures, and soforth. Surface painter 106(P) is capable of applyingspecially-identified irregular features to a target surface 102. Surfacepainter 106(P), in accordance with user-specified constraints,synthesizes irregular features as a foreground texture such that theyare realistically merged with a background texture.

A target surface 102 is obtained, and a selected regular BTF sample104(R) is applied thereto. Surface tiler 106(T) tiles or synthesizesregular BTF sample 104(R) onto target surface 102 to produce tiledsurface 108. Tiled surface 108 can be further enhanced or augmented withirregular BTF sample 104(I), which can be imperfections or other specialforeground features. Surface painter 106(P) paints or synthesizesirregular BTF sample 104(I) onto tiled surface 108 to produce tiled andpainted surface 110.

Although implementations are primarily described herein with regard toBTF type textures, other textures may alternatively be utilized. Also, asingle texture sample 104 may include both regular and irregularaspects.

FIG. 2 illustrates example blocks relating to surface tiling with aregular texture sample 104(R) and surface painting with an irregulartexture sample 104(I). Target surface 102 has an original or inputsurface mesh 202. In a described implementation, a dense surface mesh204 is constructed using a texture atlas 206. For example, a multi-chartgeometric image mesh (MCGIM) may be used along with texture atlas 206 toresample input surface mesh 202 to produce dense surface mesh 204. Themesh is partitioned into fairly large charts that are parameterized withminimized stretch. Consequently, mesh patches 208 of dense surface mesh204 may be developed that are analogous to image charts 210 of textureatlas 206.

During surface tiling, work patches 212 from dense surface mesh 204 areidentified, with each work patch 212 being associated with a respectivework image 214 or portion thereof from image charts 210. As a result ofthe re-sampling that constructs dense surface mesh 204, vertices 216 ofdense surface mesh 204 correspond to grid points or pixels 218 of imagecharts 210. More specifically, there is a one-to-one correspondencebetween each vertex 216 of dense surface mesh 204 and each pixel 218 ofimage charts 210. It should be understood that image resolution, ratiorealities, mathematical remainders, etc. can prevent the creation of a100% one-to-one correspondence; nevertheless, a substantially one-to-onecorrespondence can usually be created.

As a consequence of the one-to-one correspondence, some operations canbe performed in the image space on image charts 210 with pixels 218 andother operations can be performed in the mesh space on mesh patches 208with vertices 216. Moreover, results can be moved between the imagespace and the mesh space.

In a described implementation, for example, patch matching 222 isperformed and accomplished in the image space with respect to a workimage 214. To account for the three-dimensional (3D) nature of theoriginal input surface mesh 202, patch matching may involve rotations aswell as translations. Patch fitting 220, on the other hand, is performedand accomplished in the mesh space with respect to a work patch 212. Thepatch fitting may be effectuated with a graphcuts operation.

Feature painting 224 is part of surface painter 106(P). Hence, anirregular texture feature is applied to an input surface mesh 202. Agraphcuts operation is employed to effectuate the feature painting.However, in a described implementation, the graphcuts operation isconstrained by user input(s). For example, a user may specifyconstraints as to a desired foreground feature of a sample texture104(I) and an intended foreground region of a targeted surface 102. Inan example embodiment, the constraint energy as defined by the user isminimized as part of the graphcut operation.

FIG. 3 is a flow diagram 300 that illustrates an example of a method fortiling a surface with a regular texture sample. Flow diagram 300includes seven (7) blocks 302-314. Although the actions of flow diagram300 may be performed in other environments and with a variety ofhardware and software combinations, FIGS. 1-2 are used in particular toillustrate certain aspects and examples of the method. By way of exampleonly, the actions of flow diagram 300 may be performed by a surfacetiler 106(T) of a surface decorator 106.

At block 302, a dense mesh is constructed from an input target mesh suchthat mesh vertices of the dense mesh have a one-to-one correspondencewith pixels of an image. For example, dense surface mesh 204 may beconstructed from input surface mesh 202 (of a target surface 102) suchthat vertices 216 have a one-to-one correspondence with pixels 218. Thisconstruction may be effectuated using, for instance, a multi-chartgeometric image mesh (MCGIM) derived from target surface 102.

At block 304, a work patch is built around a vertex of the dense mesh.For example, a work patch 212 may be built around a most-constrainedvertex 216 of vertices 216 of dense surface mesh 204. At block 306, awork image is created from a work patch using image charts. For example,work image 214 may be created from work patch 212 using image charts 210that are derived responsive to a texture atlas 206.

At block 308, a sample patch is matched to pre-existing pixels in thework image using rotations as well as translations of the sample patch.For example, a regular texture sample 104(R) may be matched 222 topixels that have already been added to work image 214. The matchinginvestigation may utilize rotations as well as translations to find anacceptable, if not optimum, sample patch position.

At block 310, the matched sample patch is fit into the work patch usinga graph cuts algorithm. For example, regular texture sample 104(R) maybe fit 220 to work patch 212 using a graphcuts procedure in dependenceon a seam cost threshold and responsive to the position located duringthe patch matching (of block 308).

At block 312, texture values are assigned to vertices based on thematched and fitted patches. For example, texture values from regulartexture sample 104(R) may be assigned to vertices 216 of dense surfacemesh 204 in accordance with the results of patch matching 222 and patchfitting 220. At block 314, the regular texture sample is synthesizedonto the originally-input target mesh. For example, regular texturesample 104(R) is therefore synthesized onto input surface mesh 202 oftarget surface 102.

FIG. 4 is a flow diagram 400 that illustrates an example of a method forpainting a surface with an irregular texture sample in accordance with auser-specified constraint function for graphcuts. Flow diagram 400includes four (4) blocks 402-408. Although the actions of flow diagram400 may be performed in other environments and with a variety ofhardware and software combinations, FIGS. 1-2 are used in particular toillustrate certain aspects and examples of the method. By way of exampleonly, the actions of flow diagram 400 may be performed by a surfacepainter 106(P) of a surface decorator 106.

At block 402, user input is accepted as demarcating a foreground featureof a texture sample. For example, an imperfection may be demarcated by auser as a foreground feature in an irregular texture sample 104(I). Atblock 404, user input is accepted as delineating a foreground region ofan output texture. For example, a region that is to receive theirregular feature of irregular texture sample 104(I) may be delineatedby a user as a foreground region of the ultimate output texture ontarget surface 102. The input foreground feature and foreground regionare user constraints for the irregular feature painting process 224.

At block 406, graphcuts are implemented as constrained by theuser-specified foreground feature and region. For example, a graphcutsoperation is effectuated responsive to the demarcated irregular featureof irregular texture sample 104(I) and the delineated foreground regionof the ultimate output texture on target surface 102.

In a described implementation and as indicated by block 406A, thisgraphcuts operation is further effectuated by minimizing the constraintenergy. The constraint energy measures how well the synthesized texturesatisfies the user-specified constraint. Moreover, both the smoothnessenergy and the constraint energy may be minimized during the graphcutsoperation. These energies and a graphcuts operation for irregulartexture painting are described further herein below with particularreference to FIGS. 10 and 11.

At block 408, the irregular texture sample is synthesized onto theoriginally-input target mesh. For example, irregular texture sample104(I) is therefore synthesized onto input surface mesh 202 of targetsurface 102. For improved blending and aesthetic appearance, inputsurface mesh 202 in this context can comprise a tiled surface 108.

Quantitative Description of Decorating Surfaces with Texture SamplesFoundation

With respect to texture synthesis, algorithms for synthesizing textureson surfaces can be divided into two categories. A first category isbased on per-pixel non-parametric sampling. However, per-pixel samplingis susceptible to the problems caused by the fact that the commonly-usedL2-norm is actually a poor measure of perceptual similarity.Consequently, algorithms in this first category have difficultymaintaining texture patterns with certain types for textures. Althoughthere have been remedies proposed for various specific scenarios, thereis no prior general solution.

Algorithms in a second category synthesize textures by copying patchesof the input texture sample. Because texture patterns are directlycopied onto the target surface, these second category algorithms are notseriously affected by the problems related to the L2-norm. Earliersecond category algorithms randomly paste patches and use alpha-blendingto hide patch seams. Quilting, however, generates significantly betterresults by carefully placing patches to minimize the discontinuityacross patch seams. After placing patches, in addition to usingalpha-blending to hide patch seams, it is possible to further enhancethe smoothness across the seams by searching for the “min-cut” seams. Ithas been shown that, for image textures, quilting with graph cutsproduces arguably the best visible results on the largest variety oftextures. Implementations disclosed herein describe how to perform graphcuts on surfaces.

With respect to decorating surfaces with imperfections, it has beenproposed to use a set of texturing operations that include tiling andpositioning of small images onto surfaces. Existing techniques generallyfocus on generating synthetic surface imperfections. Implementationsdisclosed herein describe how to synthesize imperfections fromreal-world as well as synthetic samples.

Introduction

Described herein are schemes, mechanisms, techniques, etc. that enablethe decoration of arbitrary surfaces with BTFs. Two relativelyhigh-level texturing operations are supported: tiling and painting.Given a BTF sample, the tiling operation can automatically synthesize aBTF that fits the target surface naturally and seamlessly. The BTF canmodel spatially-variant reflectance and mesostructures. Moreover, theBTF can be derived (e.g., measured) from real materials. Hence, thetiling operation provides a convenient way to cover a surface withfairly realistic textures. FIG. 5A shows an example of a surface thathas been tiled with a relatively realistic texture. The paintingoperation further enhances realism by introducing irregular features andother imperfections. FIG. 5B shows an example of the tiled surface ofFIG. 5A after being augmented with an irregular feature. In other words,the painting operation has introduced surface imperfections into thehomogeneous BTF.

The painting operation is valuable because BTFs generated by describedsynthesis algorithm(s), as well as by other synthesis algorithms, aretypically homogeneous across the whole decorated surface. With thepainting operation, the global homogeneity can be broken by addingirregular local features. In particular, imperfections of real materialscan be captured and then painted onto the surface such that the paintedimperfections fit in seamlessly with the (e.g., tiled) backgroundpatterns.

Three-dimensional (3D) painting in general is a well-establishedtechnique for creating patterns on surfaces. BTF painting extendstraditional techniques in two ways. First, it provides a way to achievesuperior realism by applying BTFs that are measured from real materials.Second, BTF painting reduces the demand on artistic talents and thetedium that is needed for creating realistic imperfections.

Two primary challenges were addressed in the development of thedescribed system. First, BTF synthesis on surfaces remains difficult formany BTF samples. Typically, the most difficult problem is maintainingthe mesostructures of the input BTF sample. An existing algorithmaddresses this problem with partial success by synthesizing the BTFpixel-by-pixel. However, synthesizing a BTF pixel-by-pixel leads tofundamental problems in maintaining mesostructures because L2-norm is apoor measure of perceptual similarity.

An alternative is to synthesize BTFs by copying patches of the inputsample (i.e., to employ quilting). Because mesostructures are copiedalong with the patches, this approach is particularly effective formaintaining mesostructures. Unfortunately, patch seams still present aproblem for BTFs being applied with patches. Although techniques existfor hiding seams in surface textures, these techniques do not generalizewell to the mesostructure-containing BTFs. For example, blending can beused to hide patch seams, but such blending creates inconsistentmesostructures in the blended areas.

Described herein are algorithm(s) for quilting BTF patches by usinggraphcuts to minimize the discontinuity across patch seams on arbitrarysurface meshes. A straightforward extension and application of graphcutsto hierarchical pattern mapping can be used to generate texturecoordinates for each triangle of an arbitrary surface mesh. However,such an approach leads to textures that reveal patch seams, at leastwhen viewed close up. This is a potential problem for any attempt toapply graphcuts to surface meshes.

In described implementation(s), this problem is addressed by denselyre-sampling the surfaces using geometry images. This approach is termed“GIM-based sampling” herein. Specifically, given an input mesh, a densemesh is created by densely sampling the input mesh using multi-chartgeometric images (MCGIM). The texture synthesis is accomplished by usinggraphcuts to generate texture coordinates for each vertex of the densemesh. Because a texture value is computed for each vertex of the densemesh, the synthesized textures can be viewed from any distance to asimilar extent as those that are obtained with pixel-based algorithms.

FIGS. 6A-6F illustrate an example surface mesh and sample BTF in thecontext of an example of a straightforward extension of graphcuts topattern mapping as compared to an example of results of a GIM-basedsampling approach. FIG. 6A is an example of an input surface mesh. FIG.6B is an example of a BTF sample. FIGS. 6C and 6D illustrate astraightforward extension of graphcuts to pattern mapping. FIGS. 6E and6F illustrate an example of the results of a GIM-based sampling approachafter the tiling of a BTF sample.

As is apparent from FIGS. 6C and 6D, a straightforward extension ofgraphcuts to hierarchical pattern mapping leads to suboptimal quality,at least when the synthesis results are viewed from a close distance.FIG. 6D is an enlarged version of the area of FIG. 6C that is delineatedby a black box. These results are obtained by incorporating graphcutsinto pattern mapping, which works directly on the original input mesh.In FIG. 6D (and FIG. 6C, too), patch seams are visible as a disjointedzigzagged or stair-stepped border between patches.

As is apparent from FIGS. 6E and 6F, applying the GIM-based samplingapproach to BTF sample tiling results in a visually-improved texture,even from close up. FIG. 6F is an enlarged version of the area of FIG.6E that is delineated by a black box. The seams are significantly lessvisible, if visible at all, in FIG. 6F as compared to FIG. 6D.

A second primary challenge addressed in the development of the describedsystem is the difficulty of introducing irregular features into abackground pattern, especially in a user-friendly manner. Astraightforward approach with graph cut techniques is to merge aforeground feature with the background pattern. For example, the featuremay be constrained to a desired location and graph cuts may be used tofind the merging seam. However, this approach only supports verbatimcopying of existing features.

To enable a user to generate new features, in describedimplementation(s), a constrained graph cut procedure for texture and BTFpainting is presented. For texture synthesis, smoothness energy isconsidered for finding seams that minimize discontinuity. For texturepainting, on the other hand, both smoothness energy and constraintenergy are used. Consequently, a user's stipulation of the new featureis incorporated into the graph cut problem as one or more constraints.Generating a new feature with the graphcut painting algorithm is thuslyfacilitated: the user specifies the rough shape and location of thedesired new feature, and a software realization for a describedimplementation can then synthesize the actual feature and have it mergedseamlessly with the background pattern.

It should be noted that the described schemes, mechanisms, techniques,etc. are also applicable to ordinary color textures, which may beregarded as BTFs with a single viewing direction and a single lightingdirection.

BTF Synthesis or Tiling

Given a mesh M and an input BTF sample, a dense mesh M_(d) is builtfirst and then a BTF value for each vertex of M_(d) is synthesized byquilting patches of the input BTF sample on M_(d).

GIM-Based Sampling:

As explained herein above, graphcut techniques can be incorporated intoa synthesis algorithm that works directly on an original input mesh. Forexample, texture coordinates can be generated for each triangle of theinput mesh. Unfortunately, the synthesis results are not ideal, at leastwhen viewed from a close distance (as shown in FIGS. 6C and 6D). Theseams become visible with the pattern mapping approach because twoadjacent triangles on the input mesh may be far apart in the texturespace. During texture synthesis, textures on two such adjacent trianglesare matched at a fixed sampling resolution (which is a parameter of thesynthesis algorithm). When synthesis results are viewed at a higherresolution, the seam between the two triangles becomes even morenoticeable.

The GIM-based sampling approach described herein provides an effectiveand general solution for applying graphcut techniques to surfaces. Fromthe input mesh M, the dense mesh M_(d) is constructed as an MCGIM.MCGIMs are described further in “Multi-chart geometry images” by P. V.Sander, Z. Wood, S. J. Gortler, J. Synder, and H. Hoppe, in Symposium onGeometry Processing 2003, pages 146-155, 2003, which is herebyincorporated by reference in its entirety herein. The MCGIM uses atexture atlas to resample M and zippers chart boundaries to obtain a“watertight” mesh M_(d).

The texture atlas may be created, for example, using the methodsdescribed in “Iso-charts: Stretch-driven mesh parameterization usingspectral analysis” by Kun Zhou, John Synder, Baining Guo, andHeung-Yeung Shum, in Symposium on Geometry Processing 2004, pages 47-56,2004, which is hereby incorporated by reference in its entirety herein.Using these described methods for creating the texture atlas, the inputmesh is partitioned into relatively large charts, with the charts beingparameterized with minimized stretch.

FIGS. 7A-7D illustrate an example construction of a dense mesh from aninput mesh for subsequent BTF synthesis. Specifically, FIGS. 7A and 7Bshow an example texture atlas, and FIGS. 7C and 7D show an exampleMCGIM. The sampling rate of the MCGIM is a user-specified parameter.Although any values may be used for the sampling rate, example samplingrates include 512×512 and 1024×1024 (which are the values used by theexamples illustrated in the accompanying figures).

The dense mesh M_(d) may be created such that there is a one-to-onecorrespondence between vertices of M_(d) and pixels of the textureatlas. Because of this correspondence, synthesizing BTF values forvertices on M_(d) is equivalent to synthesizing BTF values for pixels ofthe charts of the texture atlas. The pixel-vertex correspondence makesit easy to flatten a large patch of M_(d) without introducing noticeabletexture distortion. In fact, for a pixel in the interior of a chart, theinterior pixel's 2D neighborhood can directly provide neighboringsamples and a local flattening. This pixel-vertex correspondence enables(i) simultaneous work on a surface patch of M_(d) and its correspondingimage and (ii) switching freely between the two.

With reference to FIGS. 7A and 7B, an example texture atlas isillustrated over a 3D rabbit. FIG. 7A shows the assembled texture atlas,and FIG. 7B shows the image patches of the texture atlas afterdisassembly. With reference to FIGS. 7C and 7D, an example multi-chartgeometry image mesh (MCGIM) is illustrated for the 3D rabbit. FIG. 7Cshows the 3D mesh formed from triangles for the rabbit, and FIG. 7Dshows the densely sampled mesh with the image patches flattened intocharts. Thus, each chart of the MCGIM corresponds to an image patch ofthe texture atlas. The charts show the surface being sampled using aregular grid. As indicated by the two dots and the double-headed arrow,each mesh vertex corresponds to a grid point.

Quilting on the Dense Mesh:

From the input of an MCGIM M_(d) and a BTF sample S, the followingpseudo-code can produce BTF values for all vertices of M_(d): Whilethere are still un-synthesized vertices in M_(d) do pick the mostconstrained un-synthesized vertex v build a work patch P(v) centered atv found_work_patch = FALSE While found_work_patch == FALSE do If P(v)lies inside a single chart found_work_patch = TRUE obtain the work imageI(v) from the chart Else parameterize P(v) using LSCM Ifparameterization_distortion ≦(distortion) threshold T_(d)found_work_patch == TRUE create the work image I(v) by resampling P(v)Else decrease the size of P(v) found_cut = FALSE While found_cut ==FALSE do patch matching in I(v) patch fitting in P(v) using graph cutsIf the optimal seam cost is below (seam cost) threshold T_(c) found_cut= TRUE Else decrease the size of I(v) and P(v) get BTF values forvertices of P(v)The above algorithm can implement texture quilting on the dense mesh.

The algorithm synthesizes the surface BTF by copying patches of thesample BTF. Each time, the next BTF patch P_(b) that is to be generatedis around the un-synthesized vertex v which is the most constrained. Themost constrained vertex is the vertex v having the largest number ofimmediate vertex neighbors that are already synthesized.

Work Patch/Image: To determine the next BTF patch P_(b), a work patchP(v) centered at v is built first. A breadth-first traversal isconducted starting at v to ascertain P(v). The number of levels in thebreadth-first traversal is defined as a user-supplied parameter calledthe work patch size r_(w). For a 512×512 MCGIM M_(d), for example, atypical working patch size is r_(w)=32, but other working patch sizesr_(w) may be selected. Intuitively, the work patch P(v) may be thoughtof as a disk of radius r_(w).

From the work patch P(v), a work image I(v) may be derived using acontinuous parameterization of P(v) by considering two cases. The firstcase is simpler: If the work patch P(v) lies completely inside a chartof the MCGIM M_(d), then the work image I(v) is the correspondingsub-image of the chart image, and no parameterization or resampling isnecessary. The second case is more complex: If the work patch P(v)crosses one or more chart boundaries, then P(v) is parameterized using aleast squares conformal mapping (LSCM) technique, and the result isresampled to obtain the work image I(v). An example of an LSCM techniqueis provided in “Least squares conformal maps for automatic texture atlasgeneration” by B. L'evy, S. Petitjean, N. Ray, and J.-L. Mallet, inProceedings of SIGGRAPH 2002, pages 362-371, 2002, which is herebyincorporated by reference in its entirety herein.

To reduce, if not minimize, the texture distortion caused by LSCM, thearea distortion is monitored by a distortion threshold T_(d). Althoughother values may be used, an example value for distortion thresholdT_(d) is 4. For resampling, the sampling step is set to the average edgelength of the work patch.

If a sampling point p is located in a triangle with three synthesizedvertices, the BTF value at point p is interpolated from these threevertices and the sampling point is marked as synthesized. Otherwise, thepoint p is marked as un-synthesized. In this manner, apartially-synthesized work image T(v) is obtained.

FIGS. 8A-8D illustrate an example BTF synthesis using a work patch and awork image. FIG. 8A shows the finding of a work patch on the targetsurface. FIG. 8B shows the initial work patch. FIG. 8C shows the workimage in a state in which there are synthesized and un-synthesizedpixels. FIG. 8D shows the work patch after the fitting of a new texturepatch; the graph cut seam is also specifically indicated.

After building the work patch and image, the next BTF patch P_(b) can bedetermined using graph cuts. For quilting with graph cuts, the maintasks are patch matching and patch fitting. Patch matching places acandidate patch (e.g., the input sample) over a target area by comparingthe candidate patch and the synthesized pixels in the target area. Patchfitting applies graph cuts to select a piece of the candidate patch tobe the next synthesized BTF patch P_(b).

Patch Matching: Patch matching is performed in the work image I(v). Anexample of a suitable general patch-matching technique is presented in“Graphcut textures: Image and video synthesis using graph cuts” by VivekKwatra, Arno Schödl, Irfan Essa, Greg Turk, and Aaron Bobick, in ACMTransactions on Graphics, 22(3):277-286, July 2003, which is herebyincorporated by reference in its entirety herein. Thesum-of-squared-differences (SSD) cost for the BTF is normalized withinthe area of the overlap region between the input sample and the workimage, and the patch placement with the minimal cost is selected forpatch fitting. The SSD-based search may be accelerated using FastFourier Transform (FFT).

However, patch matching need not be limited solely to translation as ingeneral approaches. In described implementation(s), instead of relyingon only translations, patch matching employs both translations androtations. The rotation capability helps to handle the 3D nature of theoriginal target surface.

In other words, surface quilting involves a vector field, and thedescribed rotational aspect of patch matching accommodates thedifference. The vector field aides in orienting anisotropic textures onsurfaces and for BTF rendering, which involves a local frame at eachsurface point. In an implementation of a software program for BFTtexture synthesis, a user interface is provided to let the user specifyvectors at a few relatively “key” faces. These vectors are theninterpolated to other faces using radial basis functions.

During patch matching, an average orientation for the work image iscalculated using the vector field on the work patch. The averageorientation is then used to align the input BTF sample S for patchmatching. For computational efficiency, a predetermined number ofrotated versions of S may be pre-computed. (For the illustratedexamples, 72 pre-computed rotation versions are used.) During patchmatching, the best pre-computed rotated version of S is selected for useduring patch fitting.

Patch Fitting: After the input sample S is placed over the work imageI(v) by patch matching, the input sample S can be mapped onto the workpatch P(v) using the correspondence between I(v) and P(v) (i.e., eachgrid point or pixel of work image I(v) corresponds to a vertex of workpatch P(v)).

Each vertex of the work patch P(v) that is covered by the mapped androtated input sample S is assigned a new BTF value. Eachalready-synthesized vertex of the work patch P(v) also has an old BTFvalue. Based on these old and new BFT values, a graph cuts technique isapplied to select a piece of the input sample S by regarding thevertices of the work patch P(v) as graph nodes and the edges of the workpatch P(v) as graph edges. The new BTF values of the selected piece ofthe input sample S are copied or assigned to the vertices of the workpatch P(v) if the seam cost is below a predetermined cost thresholdT_(c). It should be noted that the Kwatra document as referencedhereinabove presents an approach for accounting for old seams fromprevious graph cut steps.

Hierarchical Search: To achieve relatively higher-quality results, apiece of the input sample S that is selected by the graph cuts techniqueis utilized if the seam cost is below a seam cost threshold T_(c)If theseam cost exceeds the seam cost threshold T_(c), the work patch sizer_(w) is decreased and the search process is repeated. For example, thework patch size r_(w) may be reduced by a constant factor, and then thesearch for another, better piece of the input sample S is commenced.(For the illustrated examples, the constant reduction factor is set to0.8.)

The work patch size r_(w) may be decreased one or more times until theseam cost falls below the seam cost threshold T_(c), or until the workpatch size r_(w) reaches a minimum size. (For the illustrated examples,the minimum size of the work patch size r_(w) is 4.) The samehierarchical search strategy can be employed to control the areadistortion of the LSCM parameterization when the work patch is built.

Handling BTF Values: The BTF sample is a six dimensional (6D) functionf(x, v, l), where x=(x, y) is the texture coordinate. v=(θ_(v), φ_(v))and l=(θ_(l), φ_(l)) are the viewing and lighting directions,respectively, in spherical coordinates.

For each texel x=(x, y), the BTF value is a 4D function that isdiscretized into a high dimensional vector. Careful treatment of thishigh dimensional vector can be beneficial when storing BTFs and/orcalculating SSD costs during patch matching and patch fitting. Forexample, for efficient storage of various BTFs, including rotated copiesof the input sample S and the synthesized surface BTFs, storage can belimited to the texture coordinates. Upon request, the actual BTF valueis retrieved from the input sample S using texture coordinates.

For efficient SSD cost calculation, the BTF may be factorized as a sumof products of 2D and 4D functions using singular value decomposition(SVD). An example equation representing such an SVD is:${{f\left( {x,v,l} \right)} \approx {\sum\limits_{i = 1}^{n}{g_{i}\quad(x)\quad p_{i}\quad\left( {v,l} \right)}}},$where g_(i)(x) is termed a geometry map and p_(i)(v, l) is termed aneigen point appearance function (PAF). FIG. 9 illustrates an example SVDfactorization of a BTF sample. More specifically, it is an example SVDfactorization of a 128×128 BTF sample with the viewing and lightingresolution equal to 12×8×12×8. FIG. 9(a) shows two images of a sampleBTF with different viewing and lighting directions. FIGS. 9(b), 9(c),and 9(d) show three relatively significant eigen PAFs (along the toprow) and geometry maps (along the bottom row). A 4D PAF is packed intothe 2D image of FIG. 9; each row corresponds to a varying lightdirection, and each column corresponds to a varying view direction.

The geometry maps depend on texture coordinates only, but the PAFs are afunction of the viewing and lighting directions. An algorithm thatsynthesizes a surface BTF using the corresponding geometry maps may beused. In fact, for the illustrated examples, this algorithm is adoptedfor surface quilting with n=40, and the SSD costs are calculated withlow-dimensional vectors.

For ordinary color textures, vertex colors are stored directly intoM_(d) to thereby create a texture map (e.g., usually of a size of512×512 or 1024×1024 for the examples included herein). With thiscreated texture map, the synthesized surface texture can be renderedusing the standard texture mapping pipeline.

BTF Painting

BTF painting can be applicable, for example, in the following scenario:A sample texture S has a foreground feature over a background pattern.The background pattern is already synthesized onto a target mesh M. Inaccordance with described implementation(s), a BTF painting systemenables a user to interactively synthesize new foreground features onthe target mesh M by roughly specifying their shapes and locations. Inthis scenario, it is assumed that prior to the foreground synthesis theuser has roughly specified the foreground feature(s) in the BTF sampleS.

As noted herein above in the qualitative section, a describedimplementation of the painting system utilizes a graphcut texturepainting algorithm that minimizes both smoothness energy and constraintenergy to build a quilt of patches.

FIGS. 10A-10D illustrate the construction or painting of a constrainedquilt for an irregular feature for a foreground texture. FIG. 10A showsan input texture sample. The input texture may be considered to have aregular texture portion and an irregular texture portion. FIG. 10B showshow a foreground feature may be specified by a user. The foregroundfeature is the white portion of the input texture sample and isconsidered to be an irregular portion in this example. FIG. 10C shows aforeground region that is specified by a user for the output texture.The foreground region is the white portion. FIG. 10D shows the resultingconstructed constrained quilt. The graphcut patch boundaries are alsoindicated.

Graphcut Texture Painting:

The graphcut texture painting in this example results in theconstruction of a constrained quilt Q. The input sample texture S hasbeen partitioned into two parts by a user: a background pattern S_(b)and a foreground feature S_(f). In the target mesh M, the user has alsospecified a foreground region F. The constrained quilt Q(F) is a quiltconsisting of patches of M such that Q(F) covers F and each patch ofQ(F) is directly texture-mapped from a patch in the input sample S. Theconstraint on Q(F) is that F is textured by the foreground featureS_(f).

To construct the constrained quilt Q(F) using graph cuts, theuser-specified constraints are encoded as constraint functions. Afunction m₀ is defined on the target mesh M such that m₀(p)=1 if point pbelongs to F and m₀(p)=0 otherwise. Similarly, for every patch P of theconstraint quilt Q(F), a function m_(p) is defined such that m_(p)(p)=1if point p belongs to the foreground feature and m_(p)(p)=0 otherwise.With these functions defined, a constrained graph cut technique, whichis a part of graphcut texture painting, is described.

Constrained Graph Cuts: For the sake of clarity, constrained graph cutsis described for image textures. By way of explanation only, twooverlapping constrained patches A and B are addressed. Each pixel p ofpatch A has a texture value f_(A)(p) and a constraint value m_(A)(P).Similarly, each pixel p of patch B has a texture value f_(B)(p) and aconstraint value m_(B)(P). Finally, for each pixel p of the overlappingregion, there is a user-specified constraint value m₀(p).

FIG. 11 illustrates a graphical formulation of a seam finding problemfor graphcut texture painting. The top portion of FIG. 11 shows a patchA, a patch B, and a cut or overlapping region. The bottom portion showspatch A, patch B, and individual pixels of the overlapping region, aswell as links between and among them.

A goal is to assign a patch label σ_(p) to every pixel p in theoverlapping region (σ_(p)=A or B), so that the region is divided intotwo parts by a seam of minimum cost (energy). The energy that isminimized is defined asE(σ)=E _(data)(σ)+E _(smooth)(σ).In general graph cut problems, the smoothness energy E_(smooth) measuresthe extent to which σ is not smooth, and the data energy E_(data)measures the difference between σ and some known data. On the otherhand, for graphcut texture painting, E_(smooth) measures how well thepatches fit together along their seams, and E_(data)=E_(constrain),measures how well the synthesized texture satisfies the user specifiedconstraint m₀.

Fitting patches together while minimizing E_(smooth) along the seams isperformed in the same manner as graphcut texture synthesis. The energyE_(smooth) is defined as${{E_{smooth}\quad(\sigma)} = {\sum\limits_{p,q}{V_{({p,q})}\quad\left( {\sigma_{p},\sigma_{q}} \right)}}},$where Σ_(p,q) is the sum over the pairs of adjacent pixels in theoverlapping region. The smoothness function V_((p,q))(σ_(p), σ_(q)) isdefined asV _((p,q))(σ_(p),σ_(q))=∥f _(σ) _(p) (p)−f _(σ) _(q) (p)∥+∥f _(σ) _(p)(q)−f _(σ) _(q) (q)∥,where f_(A)(p) and f_(B)(p) are pixel p's texture values for patch A andpatch B, respectively.

For graphcut texture painting, the user specified constraint m₀ is alsosatisfied. This is how graphcut texture painting differs from graphcuttexture synthesis, which uses E_(smooth) but not the user specifiedconstraint m₀. The user specified constraint m₀ is incorporated intographcut texture painting or quilting by making use of the energyE_(data). The energy E_(data) is defined as${{E_{data}\quad(\sigma)} = {{E_{constraint}\quad(\sigma)} = {\sum\limits_{p}{D_{p}\quad\left( \sigma_{p} \right)}}}},$where Σ_(p) is the sum over the pixels in the overlapping region. Thefunction D_(p)(σ_(P)) is defined asD _(p)(σ_(p))=∥m _(σ) _(p) (p)−m ₀(p)∥,where m_(σ) _(p) (p) is the constraint value of patch σ_(p) at pixel p,while m₀(p) is the user specified constraint value at pixel p.

An optimal swap move approach is used to minimize the energy E(a). Anexample of a suitable optimal swap move technique is presented in “Fastapproximate energy minimization via graph cuts” by Yuri Boykov, OlgaVeksler, and Ramin Zabih, in IEEE Trans on Pattern Analysis and MachineIntelligence, 23(11):1-18, November 2001, which is hereby incorporatedby reference in its entirety herein.

The bottom portion of FIG. 11 illustrates a graph construction forimplementing an optimal swap move technique. The nodes of the graph are(i) the pixels (numbered 1-9) in the overlapping region and (ii) the twoterminals A and B. Each pixel p in the overlapping region is connectedto the terminals A and B by edges t_(p) ^(A) and t_(p) ^(B),respectively. These edges are referred to as t-links (terminal links).Each pair of the adjacent pixels (p,q) in the overlapping region isconnected by an edge e(p,q) called an n-link (neighbor link). Theweights of the edges areweight(t _(p) ^(A))=D _(p)(A),weight(t_(p) ^(B))=D _(p)(B), andweight(e _((p,q)))=V _((p,q))(A,B)

Applying the min-cut algorithm to the constructed graph produces theminimum cost cut that separates node A from node B. In a describedimplementation, any cut in the graph is to include exactly one t-linkfor any pixel p. Thus, any cut leaves each pixel p in the overlappingregion with exactly one t-link, which defines a natural label σ_(p)according to the minimum cost cut C, such that σ_(p) is$\sigma_{p} = \left\{ {\begin{matrix}A & {t_{p}^{A} \in C} \\B & {t_{p}^{B} \in C}\end{matrix}.} \right.$The approach for handling old seams in graphcut texture synthesis alsoworks with graphcut texture painting.

Quilting on Surfaces: The surface quilting algorithm described hereinabove in the context of graphcut texture synthesis can be adopted forgraphcut texture painting on surfaces with modifications to the patchmatching and patch fitting phases. For patch matching, both the BTFvalues and the constraint values (the m_(s) and m₀ values) are used toperform a joint search for optimal patch placement. For patch fitting, aconstrained graph cut is employed.

Painting System Software Example

Implementation(s) as described herein can be realized as a paintingsystem software program that has, for example, a graphical userinterface (GUI). For user interactivity reasons, surfaces are notimmediately rendered with the BTF. Instead, an ordinary color texture isdisplayed that provides a quick preview of the actual BTF. The colortexture is obtained from one of the BTF images (e.g., the one with thefront parallel view and head-on lighting).

The example painting system modifies the user-specified constraintfunction m₀ to improve the quality of the constrained quilt. Asdescribed above, the user specifies the foreground region F on thetarget surface and thus defines the constraint m₀ such that that m₀=1over F and m₀=0 elsewhere. Unfortunately, m₀ defined in this way canresult in an abrupt change along the boundary of F. This abrupt changeoften leads to destroyed texture patterns in nearby areas.

FIG. 12 illustrates an input texture sample and resulting decoratedsurfaces under different example constraint function m₀ scenarios. FIG.12(a) shows the input texture sample. FIG. 12(b) shows the demarcationof the user-specified foreground feature of the input texture sample.FIG. 12(c) shows the delineation of the user-specified foreground region(e.g., the user-specified constraint function m₀) for the outputtexture. FIG. 12(f) shows how defining m₀ as described above and asshown in FIG. 12(c) can result in an abrupt change along the boundary ofF, which can deteriorate adjacent texture areas. It is apparent thatthere are broken texture patterns in the orange area.

One (relatively naive) approach to ameliorating this problem is toweaken m₀ as m₀=λ over F and m₀=0 elsewhere for some small λ. From thedefinition of the data energy E_(data), if m₀ is set to a small value λ,E_(data) plays a less important role in the total objective energy.However, when λ is small, m₀ ceases to be fully effective, and somebackground elements may start to appear in F as shown in FIG. 12(e).

Another approach that reduces if not eliminates the appearance ofbackground elements in the foreground is as follows: A transition zonefrom the boundary of F is expanded, and the values of m₀ in thetransition zone are interpolated using linear interpolation. Thetransition zone with an interpolated constraint function m₀ is shown inFIG. 12(d) as a blurring of the black and white division between thebackground and the foreground. Optionally, an example of painting systemsoftware may permit the width of the transition zone to be auser-adjustable parameter. FIG. 12(g) shows a synthesis result when atransition zone established by an interpolated constraint function m₀ isemployed.

In addition to synthesizing new foreground features, an example of thepainting system software also supports verbatim copying of theforeground feature from the input sample onto the target surface. Anoptional useful feature of the example painting system provides previewswith verbatim copying operations by directly projecting a stipulatedforeground feature onto the targeted surface. Although the projectedforeground feature is not seamlessly merged with the background, thepreview still provides valuable visual feedback. Thus, the user canslide the foreground feature around the surface to find a desiredpasting location; the foreground feature can then be mapped onto thesurface relatively quickly using the work patch/image at the targetlocation.

The devices, actions, aspects, features, functions, procedures, modules,data structures, components, etc. of FIGS. 1-12 are illustrated indiagrams that are divided into multiple blocks. However, the order,interconnections, interrelationships, layout, etc. in which FIGS. 1-12are described and/or shown is not intended to be construed as alimitation, and any number of the blocks can be modified, combined,rearranged, augmented, omitted, etc. in any manner to implement one ormore systems, methods, devices, procedures, media, apparatuses, APIs,arrangements, etc. for decorating surfaces with textures. Furthermore,although the description herein includes references to specificimplementations (including a general device of FIG. 13), the illustratedand/or described implementations can be implemented in any suitablehardware, software, firmware, or combination thereof and using anysuitable texture sample(s), graphcut algorithm(s), meshdensity-increasing mechanism(s), patch fitting technique(s), userinterface approach(es), and/or energy minimization procedure(s), and soforth.

Example Operating Environment for Computer or Other Device

FIG. 13 illustrates an example computing (or general device) operatingenvironment 1300 that is capable of (fully or partially) implementing atleast one system, device, apparatus, component, arrangement, protocol,approach, method, procedure, media, application programming interface(API), some combination thereof, etc. for decorating surfaces withtextures as described herein. Operating environment 1300 may be utilizedin the computer and network architectures described below.

Example operating environment 1300 is only one example of an environmentand is not intended to suggest any limitation as to the scope of use orfunctionality of the applicable device (including computer, networknode, entertainment device, mobile appliance, general electronic device,etc.) architectures. Neither should operating environment 1300 (or thedevices thereof) be interpreted as having any dependency or requirementrelating to any one or to any combination of components as illustratedin FIG. 13.

Additionally, implementations for decorating surfaces with textures maybe realized with numerous other general purpose or special purposedevice (including computing system) environments or configurations.Examples of well known devices, systems, environments, and/orconfigurations that may be suitable for use include, but are not limitedto, personal computers, server computers, thin clients, thick clients,personal digital assistants (PDAs) or mobile telephones, watches,hand-held or laptop devices, multiprocessor systems,microprocessor-based systems, set-top boxes, programmable consumerelectronics, video game machines, game consoles, portable or handheldgaming units, network PCs, videoconferencing equipment, minicomputers,mainframe computers, network nodes, distributed or multi-processingcomputing environments that include any of the above systems or devices,some combination thereof, and so forth.

Implementations for decorating surfaces with textures may be describedin the general context of processor-executable instructions. Generally,processor-executable instructions include routines, programs, protocols,objects, functions, interfaces, components, data structures, etc. thatperform and/or enable particular tasks and/or implement particularabstract data types. Realizations of surface decorating with textures,as described in certain implementations herein, may also be practiced indistributed processing environments where tasks are performed byremotely-linked processing devices that are connected through acommunications link and/or network. Especially but not exclusively in adistributed computing environment, processor-executable instructions maybe located in separate storage media, executed by different processors,and/or propagated over transmission media.

Example operating environment 1300 includes a general-purpose computingdevice in the form of a computer 1302, which may comprise any (e.g.,electronic) device with computing/processing capabilities. Thecomponents of computer 1302 may include, but are not limited to, one ormore processors or processing units 1304, a system memory 1306, and asystem bus 1308 that couples various system components includingprocessor 1304 to system memory 1306.

Processors 1304 are not limited by the materials from which they areformed or the processing mechanisms employed therein. For example,processors 1304 may be comprised of semiconductor(s) and/or transistors(e.g., electronic integrated circuits (ICs)). In such a context,processor-executable instructions may be electronically-executableinstructions. Alternatively, the mechanisms of or for processors 1304,and thus of or for computer 1302, may include, but are not limited to,quantum computing, optical computing, mechanical computing (e.g., usingnanotechnology), and so forth.

System bus 1308 represents one or more of any of many types of wired orwireless bus structures, including a memory bus or memory controller, apoint-to-point connection, a switching fabric, a peripheral bus, anaccelerated graphics port, and a processor or local bus using any of avariety of bus architectures. By way of example, such architectures mayinclude an Industry Standard Architecture (ISA) bus, a Micro ChannelArchitecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video ElectronicsStandards Association (VESA) local bus, a Peripheral ComponentInterconnects (PCI) bus also known as a Mezzanine bus, some combinationthereof, and so forth.

Computer 1302 typically includes a variety of processor-accessiblemedia. Such media may be any available media that is accessible bycomputer 1302 or another (e.g., electronic) device, and it includes bothvolatile and non-volatile media, removable and non-removable media, andstorage and transmission media.

System memory 1306 includes processor-accessible storage media in theform of volatile memory, such as random access memory (RAM) 1310, and/ornon-volatile memory, such as read only memory (ROM) 1312. A basicinput/output system (BIOS) 1314, containing the basic routines that helpto transfer information between elements within computer 1302, such asduring start-up, is typically stored in ROM 1312. RAM 1310 typicallycontains data and/or program modules/instructions that are immediatelyaccessible to and/or being presently operated on by processing unit1304.

Computer 1302 may also include other removable/non-removable and/orvolatile/non-volatile storage media. By way of example, FIG. 13illustrates a hard disk drive or disk drive array 1316 for reading fromand writing to a (typically) non-removable, non-volatile magnetic media(not separately shown); a magnetic disk drive 1318 for reading from andwriting to a (typically) removable, non-volatile magnetic disk 1320(e.g., a “floppy disk”); and an optical disk drive 1322 for reading fromand/or writing to a (typically) removable, non-volatile optical disk1324 such as a CD, DVD, or other optical media. Hard disk drive 1316,magnetic disk drive 1318, and optical disk drive 1322 are each connectedto system bus 1308 by one or more storage media interfaces 1326.Alternatively, hard disk drive 1316, magnetic disk drive 1318, andoptical disk drive 1322 may be connected to system bus 1308 by one ormore other separate or combined interfaces (not shown).

The disk drives and their associated processor-accessible media providenon-volatile storage of processor-executable instructions, such as datastructures, program modules, and other data for computer 1302. Althoughexample computer 1302 illustrates a hard disk 1316, a removable magneticdisk 1320, and a removable optical disk 1324, it is to be appreciatedthat other types of processor-accessible media may store instructionsthat are accessible by a device, such as magnetic cassettes or othermagnetic storage devices, flash memory, compact disks (CDs), digitalversatile disks (DVDs) or other optical storage, RAM, ROM,electrically-erasable programmable read-only memories (EEPROM), and soforth. Such media may also include so-called special purpose orhard-wired IC chips. In other words, any processor-accessible media maybe utilized to realize the storage media of the example operatingenvironment 1300.

Any number of program modules (or other units or sets ofprocessor-executable instructions) may be stored on hard disk 1316,magnetic disk 1320, optical disk 1324, ROM 1312, and/or RAM 1310,including by way of general example, an operating system 1328, one ormore application programs 1330, other program modules 1332, and programdata 1334. These processor-executable instructions may include, forexample, one or more of: a surface decorator; a surface tiler; a surfacepainter; a target surface; a BTF sample; a tiled surface; a tiled andpainted surface; patches, images, or other intermediate data; somecombination thereof; and so forth.

A user may enter commands and/or information into computer 1302 viainput devices such as a keyboard 1336 and a pointing device 1338 (e.g.,a “mouse”). Other input devices 1340 (not shown specifically) mayinclude a microphone, joystick, game pad, satellite dish, serial port,video camera, scanner, and/or the like. These and other input devicesare connected to processing unit 1304 via input/output interfaces 1342that are coupled to system bus 1308. However, input devices and/oroutput devices may instead be connected by other interface and busstructures, such as a parallel port, a game port, a universal serial bus(USB) port, an infrared port, an IEEE 1394 (“Firewire”) interface, anIEEE 802.11 wireless interface, a Bluetooth®) wireless interface, and soforth.

A monitor/view screen 1344 or other type of display device may also beconnected to system bus 1308 via an interface, such as a video adapter1346. Video adapter 1346 (or another component) may be or may include agraphics card for processing graphics-intensive calculations and forhandling demanding display requirements. Typically, a graphics cardincludes a graphics processing unit (GPU), video RAM (VRAM), etc. tofacilitate the expeditious display of graphics and performance ofgraphics operations. In addition to monitor 1344, other outputperipheral devices may include components such as speakers (not shown)and a printer 1348, which may be connected to computer 1302 viainput/output interfaces 1342.

Computer 1302 may operate in a networked environment using logicalconnections to one or more remote computers, such as a remote computingdevice 1350. By way of example, remote computing device 1350 may be aperipheral device, a personal computer, a portable computer (e.g.,laptop computer, tablet computer, PDA, mobile station, etc.), a palm orpocket-sized computer, a watch, a gaming device, a server, a router, anetwork computer, a peer device, another network node, or another devicetype as listed above, and so forth. However, remote computing device1350 is illustrated as a portable computer that may include many or allof the elements and features described herein with respect to computer1302.

Logical connections between computer 1302 and remote computer 1350 aredepicted as a local area network (LAN) 1352 and a general wide areanetwork (WAN) 1354. Such networking environments are commonplace inoffices, enterprise-wide computer networks, intranets, the Internet,fixed and mobile telephone networks, ad-hoc and infrastructure wirelessnetworks, mesh networks, other wireless networks, gaming networks, somecombination thereof, and so forth. Such networks and logical andphysical communications connections are additional examples oftransmission media.

When implemented in a LAN networking environment, computer 1302 isusually connected to LAN 1352 via a network interface or adapter 1356.When implemented in a WAN networking environment, computer 1302typically includes a modem 1358 or other component for establishingcommunications over WAN 1354. Modem 1358, which may be internal orexternal to computer 1302, may be connected to system bus 1308 viainput/output interfaces 1342 or any other appropriate mechanism(s). Itis to be appreciated that the illustrated network connections areexamples and that other manners for establishing communication link(s)between computers 1302 and 1350 may be employed.

In a networked environment, such as that illustrated with operatingenvironment 1300, program modules or other instructions that aredepicted relative to computer 1302, or portions thereof, may be fully orpartially stored in a remote media storage device. By way of example,remote application programs 1360 reside on a memory component of remotecomputer 1350 but may be usable or otherwise accessible via computer1302. Also, for purposes of illustration, application programs 1330 andother processor-executable instructions such as operating system 1328are illustrated herein as discrete blocks, but it is recognized thatsuch programs, components, and other instructions reside at varioustimes in different storage components of computing device 1302 (and/orremote computing device 1350) and are executed by processor(s) 1304 ofcomputer 1302 (and/or those of remote computing device 1350).

Although systems, media, devices, methods, procedures, apparatuses,techniques, schemes, approaches, procedures, arrangements, and otherimplementations have been described in language specific to structural,logical, algorithmic, and functional features and/or diagrams, it is tobe understood that the invention defined in the appended claims is notnecessarily limited to the specific features or diagrams described.Rather, the specific features and diagrams are disclosed as exemplaryforms of implementing the claimed invention.

1. A method comprising: constructing a dense mesh from a target meshsuch that vertices of the dense mesh correspond to pixels of an image;and synthesizing a sample texture onto a target surface associated withthe target mesh responsive to the dense mesh.
 2. The method as recitedin claim 1, wherein the vertices of the dense mesh have a substantiallyone-to-one correspondence with the pixels of the image.
 3. The method asrecited in claim 1, further comprising: creating a work image from awork patch using at least one image chart.
 4. The method as recited inclaim 3, further comprising: building the work patch around aconstrained vertex of the dense mesh based on a working patch size, theconstrained vertex having a largest number of immediate vertex neighborsthat are already synthesized.
 5. The method as recited in claim 3,further comprising: performing patch matching with regard to the workimage using the sample texture.
 6. The method as recited in claim 5,wherein the performing comprises: patch matching with regard to the workimage by translating and rotating the sample texture.
 7. The method asrecited in claim 3, further comprising: performing patch fitting withregard to the work patch using the sample texture and a graphcutsoperation.
 8. The method as recited in claim 1, wherein the sampletexture comprises a bidirectional texture function (BTF) sample patch.9. The method as recited in claim 1, wherein the constructing comprises:densely sampling the target mesh using one or more multi-chart geometricimage meshes (MCGIMs) in conjunction with at least one texture atlasderived from the target surface.
 10. The method as recited in claim 1,further comprising: painting an irregular feature onto the targetsurface using a graphcuts operation that is constrained by at least oneuser input.
 11. The method as recited in claim 10, wherein the at leastone user input comprises a first user input and a second user input; andfurther comprising: accepting a demarcated foreground feature of thesample texture as the first user input; and accepting a delineatedforeground region of the target surface as the second user input.
 12. Amethod comprising: performing patch matching by translating and rotatinga sample texture to locate a position for the sample texture; performingpatch fitting by implementing a graphcuts operation with the sampletexture at the located position; and assigning texture values tovertices of a target mesh responsive to the patch matching and the patchfitting.
 13. The method as recited in claim 12, wherein the patchmatching is performed in an image space, and the patch fitting isperformed in a mesh space.
 14. The method as recited in claim 12,wherein the performing patch matching comprises: performing the matchingusing at least one of a number of pre-computed rotated versions of thesample texture.
 15. The method as recited in claim 12, furthercomprising: densely resampling the target mesh to produce a dense meshwith vertices having a substantially one-to-one correspondence withpixels of at least one image; and painting an irregular feature onto thetarget mesh using a graphcuts operation that involves minimizing energycorresponding to at least one user-input constraint.
 16. A methodcomprising: accepting user input demarcating a foreground feature oftexture sample; accepting user input delineating a foreground region ofan intermediate output texture; performing a graphcuts operation thatinvolves the texture sample and that is constrained by the demarcatedforeground feature and the delineated foreground region; andsynthesizing the demarcated foreground feature onto a target mesh toproduce an updated output texture.
 17. The method as recited in claim16, wherein the performing comprises: minimizing a constraint energy,the constraint energy measuring how well a resulting synthesized texturesatisfies the foreground constraints.
 18. The method as recited in claim17, wherein the performing further comprises: minimizing a smoothnessenergy, the smoothness energy measuring how well a foreground featurepatch fits into the intermediate output texture along a seamtherebetween.
 19. The method as recited in claim 17, wherein theminimizing comprises implementing an optimal swap move algorithm onpixels that are overlapped by a foreground feature patch and theintermediate output texture.
 20. The method as recited in claim 16,further comprising: densely resampling the target mesh to produce adense mesh with vertices having a substantially one-to-onecorrespondence with pixels of an image of a target surface associatedwith the target mesh; and synthesizing another sample texture onto thetarget surface to produce the intermediate output texture.